1. Field of the Invention
This invention relates to a method for detecting an eye structure based on a figure image and its apparatus, and particularly to a detection method capable of easily detecting the upper eyelid, inner canthus and outer canthus and its apparatus.
2. Description of the Prior Art
A conventional method for detecting the upper eyelid line and inner and outer canthus points based on a figure image is described in publication "Extract of Characteristic Line from Line Drawing of Face" (by Okada, Kiriyama, Nakamura and Minami, Research Report No. 58, Kogakuin University, April 1985). This conventional method will be described below.
FIG. 8 shows an input image 801 illustrating an eye and its vicinity. The X-axis and Y-axis are the horizontal axis and vertical axis of the drawing, respectively. A primary differential operator in the X-axis and Y-axis directions having a feature corresponding to a feature of an eye is applied to the above input image 801. The primary differential operator is suitable to detect a change from lightness to darkness or from dark to lightness in an image. Picture element where the operator output has the maximum value in the X-axis and Y-axis directions is determined as an edge in each axis direction, and its output is given by an output intensity of the primary differential operator.
To each edge image in the X-axis and Y-axis directions, 8-coupled labelling is made to construct a segment. Constructing the segment with 8 coupling is a technique to judge whether the edge picture element exists in the adjacent 8 picture elements a1 to a8 when the observed picture element is determined as a0 in FIG. 9. And, the average and variance of the edge strength are obtained for each segment. When the average and variance of the edge strength are small, the segment is supposed not to construct the upper eyelid and subjected to binary coding process with an appropriate set value to obtain FIG. 10 and FIG. 11 which show an edge binary image in the X-axis and Y-axis directions.
The upper eyelid line consists of one edge line and has the shape of a hill. Concentration dispersion within a closed line formed by the segment consisting this upper eyelid line and the straight line connecting the both ends of the segment is considered to show a characteristic distribution which largely includes dark areas such as the pupil and eyelashes.
And, the inner and outer canthus points are determined to be the both end points of the aforementioned edge segment. A method for detecting the upper eyelid line and inner and outer canthus points will be described below.
(1) An upwardly convex segment is extracted from FIG. 11. Definition of the segment which is upwardly convex is made based on FIG. 12 as follows. PA1 (2) For the upwardly convex lines in FIG. 11, attention is given to one of them. A length of the segment connecting the end point where the outer canthus exists and the point on the 9th picture element from that end point is determined to be 2a. This segment is further elongated by "a" in the direction of the outer canthus to set an isosceles triangle having the base 2a and height 3a as shown in FIG. 13. And, in this triangle, a segment existing in the solid line of FIG. 7 which is below the horizontal position of the end point is extracted from the picture elements of FIG. 10. PA1 (3) The straight line which is set in the above (2) is determined to be L1. When a straight line passing through points P1 and P2 which are separated in both directions by 4 picture elements from a point of intersection of the straight line L1 and the detected lines is L2, an angle .theta. formed by L1 and L2 is obtained (see FIG. 14). PA1 (4) When the line detected in the above (2) continues from the upwardly convex line selected in the above (2), the angle .theta. must become small to some extent. Then, a threshold Th.theta. of the angle .theta. is appropriately determined to extract a line which meets the following: EQU .theta.&lt;Th.theta. (I) PA1 When there are a plurality of segments which are detected in the above (2), a segment with the minimum angle .theta. is extracted. PA1 (5) To the whole upwardly convex segment, the above (2) to (4) are applied. The treated upwardly convex segments and the segments extracted with respect to each of the treated upwardly convex segments are connected to make candidate segments for the upper eyelid line. PA1 (6) As to one of the candidate segments for the upper eyelid line, a closed line as shown in FIG. 15 is set by a straight line connecting the end points. PA1 (7) Frequency (dot parts in FIG. 15) of the part of high concentration value exceeding the set value ThG given by the following formula (II) in each closed line is determined: EQU ThG=(Gmax-Gmin)/5 (II) PA1 where, Gmax and Gmin shall be maximum and minimum concentration values in the closed line. PA1 (9) Wrinkles of the outer canthus excessively detected is removed from the detected upper eyelid line. A distance between a straight line connecting the both ends of the upper eyelid line and an optional point on a line which is perpendicular to the straight line is determined to obtain a point which is distant most. PA1 (10) From the point obtained in the above (9) and the end point on the side of outer canthus, a distance is obtained as in the above (9). The range from the longest distance to the end point on the side of outer canthus is removed (see FIG. 16). PA1 a step for examining an image having an eye and its vicinity of a human face photographed from the vertex to the jaws to detect an area positioned between a position where brightness changes from lightness to darkness and a position where brightness changes from lightness to darkness and then to lightness, and PA1 a step for detecting from the above area an area whose width is larger than the pupil's width and where an average of brightness values in the above area is darkest as the upper eyelid area. PA1 a step for examining an image having an eye and its vicinity of a human face photographed from the vertex to the jaws to detect an area positioned between a position where brightness changes from lightness to darkness and a position where brightness changes from lightness to darkness and then to lightness, PA1 a step for detecting from the above area an area whose width is larger than the pupil's width and where an average of brightness values in the above area is darkest as the upper eyelid area, PA1 a step for finding the darkest point on the above image to detect the inner canthus and outer canthus points and their adjacent dot groups which become candidates for the upper eyelid line, and PA1 a step for carrying out the logical OR between the above upper eyelid area and the above dot groups to detect the inner canthus, outer canthus and their adjacent upper eyelid line. PA1 a step for examining an image having an eye and its vicinity of a human face photographed from the vertex to the jaws to detect an area positioned between a position where brightness changes from lightness to darkness and a position where brightness changes from lightness to darkness and then to lightness, PA1 a step for detecting from the above area an area whose width is larger than the pupil's width and where an average of brightness values in the above area is darkest as the upper eyelid area, PA1 a step for finding the darkest point on the above image to detect the inner canthus and outer canthus points and a primary dot group in their vicinity which become a candidate for the upper eyelid line, PA1 a step for carrying out the logical OR between the above upper eyelid area and the above dot groups to detect the inner canthus, outer canthus and the primary upper eyelid line in their vicinity, PA1 a step for examining the above image from the vertex, detecting a point where brightness changes from lightness to darkness and carrying out the logical OR between the change point and the above upper eyelid area to detect a secondary dot group which becomes a candidate for the upper eyelid, and PA1 a step for calculating a distance between the above primary upper eyelid line and the above secondary candidate dot group in the vertex direction to detect a dot group with the smallest distance as a secondary upper eyelid line. PA1 a primary area detecting means for detecting an area positioned between a position where brightness changes from lightness to darkness and a position where brightness changes from lightness to darkness and then to lightness by examining an image having an eye and its vicinity of a human face photographed from the vertex to the jaws, PA1 an upper eyelid candidate area detecting means for detecting as the upper eyelid candidate area an area whose width is larger than a certain set value from the above area, and PA1 a secondary area detecting means for detecting as the upper eyelid area an area where an average of brightness values in the above area is darkest from the above upper eyelid candidate area. PA1 an upper eyelid area detecting means for detecting an upper eyelid area from an image having an eye and its vicinity of a human face photographed, PA1 a primary candidate detecting means for detecting a primary candidate point which is a point where brightness changes from lightness to darkness and then to lightness by examining the image from the vertex, PA1 a secondary candidate detecting means for detecting a secondary candidate point which is a point where brightness changes from lightness to darkness and then to lightness by examining the image from the direction perpendicular to the vertex direction, and PA1 a logical OR operating means for carrying out the logical OR among the above upper eyelid area detected signal, the above primary candidate point detected signal and the above secondary candidate point detected signal to output as the inner canthus, outer canthus and upper eyelid line detected signals. PA1 an upper eyelid area detecting means for detecting an upper eyelid area from an image having an eye and its vicinity of a human face photographed, PA1 an inner canthus, outer canthus and upper eyelid line detecting means for detecting inner and outer canthus points and a primary upper eyelid line from the above image, PA1 a primary detecting means for detecting a primary point where brightness changes from lightness to darkness by examining the above image from the vertex, PA1 a secondary detecting means for detecting a dot group which is a candidate for the upper eyelid area by carrying out the logical OR between the detected signal of the above upper eyelid area and the detected signal of the above primary point, and PA1 an upper eyelid detecting means for detecting a dot group with the smallest distance as a secondary upper eyelid line by calculating a distance between the above inner and outer canthus points and primary upper eyelid line and the above candidate dot group for the upper eyelid area in the vertex direction.
(a) A distance between a straight line connecting the both ends of a segment and an optional point on a segment which is perpendicular to the straight line is determined and a point P which is distant most is obtained. PA2 (b) When the longest distance obtained in the above (a) is positioned above the straight line connecting the both ends of the segment, it is defined to be convex upward.
(8) A line when frequency of the high concentration value obtained in the above (7) becomes maximum is extracted as the upper eyelid line. On the other hand, when all lines do not include the parts of high concentration value at all, setting of the candidate area of an eye is made again as the upper eyelid line is determined not to exist.
The segment obtained in the above process is determined as the upper eyelid line and its both end points as the inner canthus point and outer canthus point.
In the aforementioned conventional method, there are many parameters which have to be adjusted to smoothly operate with respect to a given image. And, the process itself is also complicated, and it is difficult to adjust the parameters when the operation is not conducted satisfactorily. Therefore, lots of troubles and time are required to accurately detect an eye structure.
Since the outer canthus point is smoothly connected with wrinkles, the edge information used here is not sufficient to judge that it is an outer canthus point. Consequently, the detection cannot be made precisely because it is affected by a change in lighting conditions.
Furthermore, although the inner and outer canthus points are obtained under the condition close to the curvature of a segment, the detected results are largely effected by the position of a double-edged eyelid.